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ABSTRACT 



A qualitative study was undertaken to examine the 
perceptions of beginning teachers who were evaluated through an innovative 
teacher evaluation process. The evaluation was the Collaborative Assessment 
Procedure (CAP) implemented in a large midwestern urban school district. 
Beginning teachers were assigned a teacher consultant who observed the 
teacher and shared the observations in conferences. The research attempted to 
determine if a link existed between the new teacher performance evaluation 
process and the beginning teachers 1 sense of efficacy with respect to 
students and other teachers. Twenty-one teachers who had participated in this 
process were interviewed. Thirteen experienced affirmation in the process and 
thought that the CAP process nurtured their professional development. Other 
participants experienced the CAP process as surveillance, and were much less 
positive about its effects. Twenty of these teachers were convinced that they 
could affect the lives of their students significantly. Fifteen spoke of a 
strong link between their CAP experiences and their growing sense of personal 
efficacy. Two gave CAP only slight credit for their increased sense of 
efficacy, and only one did not report an enhanced sense of efficacy at the 
end of the CAP year. The process was acknowledged to recognize the varying 
levels of professional development of these beginning teachers. While the 
construct of teachers' sense of efficacy remains difficult to assess, the CAP 
approach appears to enhance it for beginning teachers. (Contains 14 
references.) (SLD) 
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The importance of evaluating competence in any profession is undisputed. Yet, until 
recently, decades of poor teacher evaluation practice went virtually unchallenged. Although 
many reform proposals assume that teacher evaluation practices can fairly and consistently 
differentiate among teachers, Wise and Darling-Hammond's (1985) extensive study indicated 
that neither assumption is necessarily valid. As a result of the widespread dissatisfaction with 
existing teacher appraisal processes, new evaluation approaches are being developed in 
tandem with other school improvements efforts. Because evaluation systems are embedded in 
the school bureaucracy, significant organizational change relative to the format and structure 
of teacher evaluation is needed. With the call for school reform, the purposes of teacher 
evaluation are changing, along with resultant changes in the processes of evaluation. 

The purpose of this qualitative study was to examine the perceptions of beginning 
teachers who were evaluated through an innovative teacher evaluation process. The innovative 
teacher performance evaluation examined in this study was the Collaborative Assessment 
Procedure (CAP)* implemented in the Jones City Schools, a large midwestem urban school 
district. Beginning teachers (first year teachers and those newly hired by the district are 
technically called interns) were assigned a teacher consultant who spent numerous hours in 
direct classroom observations and conferences with the beginning teacher throughout their 
initial year in the Jones City Schools. The frequency and duration of these classroom 



*The school district, its teacher evaluation program, and all participants are pseudonyms. 



O 

ERIC 



BEST COPY AVAILABLE 



2 



observations varied depending on the CAP consultant’s caseload and how the CAP consultant 
perceived the needs of the beginning teacher. Most beginning teachers were observed 15-20 
times over the course of the academic year. Each observation included suggestions to the 
beginning teacher documented through the consultant’s written notes. Additionally, the 
consultant prepared a minimum of one interim report and a culminating final report on the 
performance of the beginning teacher. This “reinvented” beginning teacher performance 
evaluation process attempted to integrate evaluation procedures with on-going professional 
staff development through concurrent graduate classes developed specifically for those 
involved in the CAP process. 

This research attempted to determine if a link existed between the new teacher 
performance evaluation process and the beginning teachers’ sense of efficacy with respect to 
students and other teachers. The following research questions began the process of framing 
this study - a study of beginning teachers’ perceptions of their performance evaluations amidst 
complex sociocultural contexts: 

1. What were the evaluation experiences of beginning teachers in this midwestem 
urban school district? 

2. During the CAP experience, what immediate impact did the evaluation process 
have on their professional lives as perceived by these teachers? 

3 . After the CAP experience, what residual impact did the evaluation process have 
on their professional lives as perceived by these teachers? 

4. What evidence, if any, exists that the innovative teacher performance 
evaluation process (CAP) has assisted teachers in developing a sense of 
efficacy with respect to students and other teachers? 



O 




3 



3 



Design 

To gain an insider’s perspective of CAP, qualitative research techniques were 
employed. Case study analysis was selected as the appropriate method to explore the impact of 
teacher evaluation processes on beginning teachers. Underlying the research was the guiding 
assumption that although reinvention of the processes used to evaluate teachers can trigger 
change, the impact of that change is best studied through the teachers who experienced the 
process first-hand. The purpose of this study was consistent with other qualitative inquires 
where, “The intent was to engage the informants in the inquiry as collaborators in such a way 
as to surface themes and meanings that were important to them, to get a glimpse of their 
worlds, and to hear their voices” (Canning, 1992, p. 64). 

The research questions demanded an approach that was particularistic (i.e., focused on 
a bounded unit of study, the reinvention of beginning teacher evaluation processes in one urban 
school district), descriptive (i.e., designed to recount the literal experiences of these beginning 
teachers, as well as, their interpreted meanings), heuristic (i.e., intended to illuminate the 
reader’s understanding of the phenomenon under study), and inductive (i.e., based on inductive 
reasoning to form generalizations, concepts, and hypotheses emerging from data grounded in 
the context itself). 

Methodology 

The methodology of this study included a compilation of field notes, an examination of 
artifacts associated with the teachers’ CAP experience, and interviews with teachers who had 
experienced this innovative teacher evaluation process either one, two, or three years prior to 
the study. There were 21 key informants who ranged in age from the mid-twenties to the mid- 
forties. All but two key informants continue to teach in this school district of over 60,000 
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students. At the time of their CAP experiences, 13 were elementary teachers and eight were 
either middle or secondary teachers. Fifty-seven percent of the key informants were female. 
Slightly more than twice the number of European Americans were interviewed as compared to 
African Americans who participated in the study. The manner in which the key informants had 
secured teaching positions in the school district was varied, as were their educational and 
occupational histories. 

All interviews with key informants were audio-taped and interview logs were created 
for each interview, according to the process described by Merriam (1988). Descriptive themes 
or codes for the CAP experiences, as depicted by the key informants, were developed and then 
confirmed by an outside ethnographer. Triangulation was established through the use of 
different data sources and a multi-layered data base (audiotapes of interviews, examination of 
artifacts, two elite interviews, and interview logs). Drafts of individual informant profiles 
were sent to the teacher informants to afford them the opportunity to check the accuracy of 
quotations and interpretations of their interviews. Editing of the profiles occurred as a result 
of additional input through this member checking strategy. 

Themes were developed by constantly comparing one incident or unit of information 
shared by the informants with another. Both convergent and divergent strategies were used to 
construct themes that were internally homogeneous (all items in a single category were similar) 
and yet heterogeneous (differences among themes were bold and distinct). In successive 
rounds of data analysis, this procedure was repeated until the data were fully analyzed with 
respect to teachers’ sense of efficacy. Reviewing the data to support distinct themes made it 
difficult to draw the line between description and interpretation. Eisner (1991) discussed this 
process of determining themes to convey meaning: 
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...features that count in a setting do not wear labels on their sleeves: they do not 
announce themselves. Researchers must see what is to be seen, given some frame of 
reference and some set of intentions. The self is the instrument that engages the 
situation and makes sense of it.. ..it is not a matter of checking behaviors, but rather of 
perceiving their presence and interpreting their significance, (p. 33) 

The challenge in coding the data was to recognize patterns from conversations that described 
the teachers’ perceptions of their sense of teaching efficacy. 

The evaluation experiences were first described and documented within the context of 
interviews with the beginning teachers. The experiences described were then used to analyze 
the teachers’ perceptions of the immediate and residual impact of their performance evaluation 
experiences. Finally, these experiences were examined through an additional lens, that of their 
perceived impact on the beginning teachers’ sense of efficacy with respect to students and 
other teachers. 

Findings 

Explicit and Implicit Knowledge of CAP 

Explicit knowledge of the CAP process was garnered from artifacts associated with the 
process (e.g., the negotiated agreement, internal written communication to beginning teachers, 
key informants’ CAP observation and evaluation documents) associated with the process, as 
well as elite interviews with the teachers’ union president and a university consultant. These 
two champions conceptualized the CAP process and, together with the school superintendent, 
engineered its development and implementation through the district’s bureaucratic, political, 
and legal hurdles. 




Implicit knowledge of the features of the CAP process resulted from an emic 
perspective of the process. The insider’s perception was gathered from interviews with 21 
teachers who experienced the CAP process as beginning teachers. 

Perceptions of CAP: Affirmation or Surveillance 

Little neutrality surfaced in the summaries of the beginning teachers’ year-long 
evaluation experiences. A polarity of experiences emerged as either affirmation or 
surveillance. For those 13 informants who experienced affirmation, the CAP process nurtured 
their professional development through a mixture of credible feedback and viable instructional 
and management suggestions, delivered by sensitive, communicative CAP consultants. In 
contrast, surveillance was the descriptive label for the experiences reported as negative. These 
experiences produced feelings of isolation, anxiety, and self-doubt. The performance 
evaluations of these beginning teachers resulted in the perception of constant, intensely critical, 
or non-productive surveillance. Three additional key informants reported vastly different 
relationships with two CAP consultants, resulting in experiences of both affirmation and 
surveillance. 

The Implicit Features of CAP 

Another layer of data analysis revealed multiple implicit features of the CAP process, 
distinct features not identified formally to those outside the school culture, but known to the 
“insiders.” These implicit features were labeled as Luck of the Draw, High Stakes: My Job ’s 
on the Line, Cooperate and Graduate, and When It ’s over, It ’s Over. 
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The Immediate and Residual Impact of CAP on the Professional Lives of Beginning 
Teachers 

The experiences described were then used to analyze the teachers’ perceptions of the 
immediate (during the CAP experience) and residual (post-CAP) impact of their performance 
evaluation experiences. Four discreet, yet interrelated, themes captured the immediate impact 
of the CAP experience. During their first year of teaching in Jones City Schools, the CAP 
experience impacted beginning teachers by causing them to network , enculturate, comply, and 
feel a strong sense of accountability for their teaching. 

The researcher asked key informants a variety of questions to gain insight into the 
perceived residual impact of the CAP experience. Questions included inquiries such as: What 
continued relationship existed with the CAP consultant? What about the key informant’s 
current teaching was attributable to skills learned from CAP feedback? What advice would 
they offer beginning teachers embarking on their first year? What “haunted” them from the 
CAP experience? The scope of these questions revealed only one consistently discernible, 
post-CAP impact: beginning teachers felt validated by completing the CAP process 
successfully. They had documentation they met the standard set for beginning teachers in the 
Jones City Schools. 

The completion of the final CAP evaluation document marked a shift from dependence 
on the CAP consultant to dependence on newly formed peer networks and/or mentors 
established from previous experiences. “When it’s over, it’s over ” was implicitly known, just 
as graduates know the structured public graduation ceremony marks a milestone. These 
beginning teachers had been officially validated by their school system. 
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Impact of CAP on the Beginning Teachers’ Sense of Efficacy 



The final layer of data analysis addressed the research question: What evidence, if any, 
exists that the innovative teacher performance evaluation process has assisted teachers in 
developing a sense of efficacy with respect to students and other teachers? 

Ashton and Webb’s (1986) definition of teachers’ sense of efficacy served as a guide 
for this study. A review of their conceptual construct is critical to understanding the findings 
relative to the assistance of the CAP process on developing these beginning teachers’ sense of 
efficacy. They defined teachers’ efficacy as: 

...an expectancy construct.... teachers’ situation-specific expectations that they can help 
students learn. That expectation rests on assumptions of how much students are 
capable of learning what schools have to teach. Teachers’ efficacy expectations 
influence their thoughts and feelings, their choice of activities, the amount of effort they 
expend, and the extent of their persistence in the face of obstacles, (p. 3) 

Ashton and Webb outlined two discreet dimensions of efficacy: (a) sense of teaching 
efficacy, and (b) sense of personal teaching efficacy. A sense of teaching efficacy refers to 
teachers’ beliefs that they can have “an effect on student performance, despite external 
obstacles such as family background and student ability” (1986, p. 4). A sense of personal 
teaching efficacy refers to a practitioner’s individual assessment of personal teaching 
competence and personal teaching skills. A teacher with a high sense of efficacy attributes 
student success to factors within the classroom rather than to factors outside it. Rather than 
become discouraged with students’ inevitably uneven learning, teachers with a high degree of 
efficacy regard student difficulties as challenges. They expect to be able to plan creatively to 
help students overcome hurdles in acquiring new skills. On the other hand, teachers with a low 
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sense of efficacy doubt their ability to influence student learning. Teachers with little sense of 
efficacy tend to attribute lack of student progress to factors such as the students’ families, the 
school administration, the low value placed on academics in students’ environments. These 
teachers tend to reduce their effort or give up entirely with difficult students; they become 
preoccupied with their own limitations as teachers. 

The distinction between the two dimensions is important because appropriate 
professional development depends upon which dimension of efficacy is low. If teachers’ sense 
of efficacy is low because they doubt the ability of students to learn, changing those 
expectations requires evidence demonstrating that students, even low-ability students, profit 
from instruction. In contrast, if teachers’ low sense of efficacy is grounded by the belief that 
they lack the repertoire of skills needed to teach students, the sense of efficacy will be 
enhanced only if teachers acquire instructional skills that result in the opportunity to recognize 
increased student learning. Teachers’ senses of efficacy are “subjectively reasonable beliefs” 
that are maintained unless evidence is presented that challenges those beliefs (Ashton & Webb, 
1986). Understanding teachers’ sense of both teaching and personal teaching efficacy is basic 
to understanding their behaviors in the classroom. 

Teaching efficacy and CAP; Do I make a difference? 

The belief held by these beginning teachers that teaching is a worthy profession was 
supported by an equally strong belief in the efficacy of their own teaching. Key informants 
were asked to talk about the perceived difference they made in the lives of their students. 
Twenty of the 21 beginning teachers who participated in this study were convinced that they 
could significantly affect the lives of their students and they were personally committed to 
doing so. Despite the strength of the responses regarding their perceived impact on students’ 
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lives, only five of the key informants voiced a connection between their sense of teaching 
efficacy and their CAP experiences. 

One theme, “7 Knew I was Affecting Lives, ” emerged from the informants’ adamant 
belief their teaching affected the lives of their students, a conviction not dependent upon or a 
result of the CAP process. The theme, “CAP Helped Me See I Make a Difference, " recounts 
the conversations of the five informants who verbalized a link between their CAP experiences 
and their sense of teaching efficacy. The third theme, “Disillusioned, " conveys the frustration 
and weakened sense of efficacy one informant, Max, described as a result of his first and only 
year of teaching in the Jones City Schools. The final theme was titled “The Resolve to 
Survive. ” This theme was voiced so strongly it could not be ignored in the discussion of 
teaching efficacy. However, the key informants did not directly link this “resolve to survive ” 
to their CAP experiences. 

Personal teaching efficacy and CAP: Am I a better teacher? 

The results of this study also addressed the second dimension of Ashton and Webb’s 
(1986) construct of personal teaching efficacy. To ascertain the perceptions of the key 
informants, questions such as: “Did the CAP process assist you in becoming a better teacher?” 
and “How did your teaching change during your first year? Did it improve? How did you 
know?” were posed. Fifteen of these beginning teachers spoke of a strong link between their 
CAP experiences and their growing sense of personal teaching efficacy. Two key informants 
reported a more developed sense of personal teaching efficacy at the end of their first year, but 
only slightly credited the CAP experience with affecting it. While not associated with their 
CAP experiences, three other informants also believed they had developed a stronger sense of 



personal teaching efficacy during their first year in Jones City Schools. Again Max was the 
single informant who did not report an improved sense of personal teaching efficacy at the end 
of his year as a CAP participant. 



Interpretations 

Teachers’ sense of efficacy is a crucial aspect of the ethos of schools and should be 
taken into account in the assessment of school reforms such as the CAP process. Results 
obtained through this qualitative case study extend theory relative to Ashton and Webb’s 
(1986) two dimensions of teachers’ sense of efficacy. Five theoretical links between beginning 
teachers’ sense of efficacy and the CAP process emerged from the results of this study: 

(1) The CAP process demonstrated the ability of performance evaluation to affect 
beginning teachers’ sense of personal teaching efficacy. Informants reported a consistent 
link between the CAP process and their sense of personal teaching efficacy. Many of the 
discussions between these beginning teachers and their CAP consultants answered the 
teachers’ need for data to sustain and enhance their sense of personal teaching efficacy. 
Through the CAP process, conversations over multiple and varied data sources between a 
beginning teacher and his or her CAP consultant (i.e., extent and nature of student 
participation, results of teacher-developed tests, analysis of the match between lesson plans and 
courses of study) affected the novice teacher’s sense of personal teaching efficacy. 

(2) The CAP process recognizes the varying levels of professional development of 
“beginning” teachers. Evidence existed to support the proposition that CAP consultants 
attempted to be responsive to the differing needs of beginning teachers. The structure of the 
CAP process encouraged consultants to be responsive to the unfinished business of teacher 
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development in the Jones City Schools. This practice agrees with a number of researchers 
(Berliner, 1986; Fuller & Bown, 1975; Glickman, 1990; Huling-Auston, 1987; Johnston & 
Ryan, 1983; Sclan, 1993; Veenman, 1984) who believe that teachers experience different 
stages in their professional growth and those stages need to be considered when planning 
professional development activities. 

(3) Teachers with a low sense of teaching efficacy fall into patterns of conformity 
and survival teaching. One informant doubted that he affected students’ lives. This 
exceptional case agreed with Ashton and Webb’s (1986) proposition that teachers with a low 
sense of either teaching or personal efficacy fall into patterns of conformity and survivalist 
teaching. Ashton and Webb discussed the ramifications of teachers feeling a low sense of 
teaching efficacy as being evidenced in a sense of powerlessness to affect students’ lives. 

When faced with students who are not achieving, teachers with a low sense of teaching 
efficacy are unlikely to explore alternatives to their existing repertoire of instructional skills. 
These teachers define other causes for the students’ difficulties; they tend to blame poor 
student achievement on the students themselves, their parents, or the cultural environment. 

(4) Teachers’ sense of personal teaching efficacy is situation-specific. Significant 
to this discussion is the concept of “situation-specific efficacy.” Ashton and Webb (1986) 
tested Bandura’s belief that teaching efficacy fluctuates depending on several variables such as 
teachers’ perceived isolation, uncertainty about their effectiveness, lack of support and reward, 
and sense of powerlessness. The data revealed examples to support the situation-specific 
descriptor of personal teaching efficacy. For example, some informants spoke of a high sense 
of personal teaching efficacy in one school or grade level, but not in another. At times, 
informants even predicted their personal teaching efficacy-or lack of— in other settings. In 
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situations where teachers doubted their competence, discussions of almost passive compliance 
with their consultant’s suggestions ensued. Survivalist teaching strategies replaced more 
creative, divergent ones. 

(5) The construct of teachers’ sense of efficacy remains difficult to assess. 

Further effort to refine measures of teachers’ sense of efficacy will allow more defined methods 
to develop scales of high to low efficacy relative to these variables. Efficacy remains a difficult 
concept to measure because of its constant mediating effect on behavior, which in turn alters 
the sense of efficacy. 
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